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ABSTRACT 



A study was conducted to determine what factors of human 
resource development (HRD) or other learning interventions impede or enhance 
improved performance. A review of the literature suggested theories of HRD 
effectiveness, where HRD effectiveness is perceived as the extent to which 
HRD goals and objectives are achieved. Various authors state that the outputs 
of HRD can be measured at several levels, such as learning, behavior, and 
results. The research also suggests that HRD must be aligned with strategic 
planning- -although this may result in a top-down approach that will limit 
commitment from employees. A number of contextual characteristics exerting 
influence on HRD are noted, since the environment of the company influences 
the effectiveness of HRD. The study is based on data derived from a survey of 
44 Dutch companies with more than 500 employees each. The data were collected 
in 1997, using four questionnaires sent to 767 representatives of the 
companies (44 HRD representatives, 357 HRD participants, 242 supervising 
managers, and 124 of the HRD participants' subordinates) . Response rates 
ranged from 100 percent to 29 percent, depending on the category of 
participant. The study concluded that strategic HRD alignment has an impact 
on HRD effectiveness, but that factors such as problem situation, HRD 
climate, position of HRD department, and form of HRD intervention may have 
such impact only through their effect on Organization process. (Contains 29 
references.) (KC) 
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Introduction 

HRD effectiveness is apparant from the effects of HRD programmes and other learning 
interventions. However, in literature is estimated that only between 10 and 20 percent of 
capital invested in HRD and other learning interventions will lead to enduring 
performance improvement (Baldwin & Ford, 1988; Latham & Crandall, 1991; Broad & 
Newstrom, 1992; Kessels, 1996). It is therefore important to prevent these dissatisfying 
results. The research question is then: what factors impede or enhance the achieving of 
improved performance? Before answering this, it will be argued that this research fits 
within the performance paradigm of HRD and theory of HRD effectiveness, and within 
the multilevel theory. 



Multilevel performance and effectiveness of HRD 

The research question fits within the performance paradigm of HRD. According to 
Weinberger (1998) the performance paradigm of HRD holds that the purpose of HRD is 
to advance the mission of the organizational system. HRD efforts have to improve the 
capabilities of individual working in the organization and improving the organizational 
systems in which they perform their work. The primary outcome of HRD in this context 
is not just learning, but also performance at various levels (Holton, 2000). 

Kaplan and Norton (1996) suggest two categories of performance measures: the so-called 
drivers and outcomes. Outcomes measure the effectiveness or efficiency relative to core 
outputs of the system, sub-system, process or individual, whereas drivers measure 
elements of performance that are expected to sustain or increase system, sub-system, 
process, or individual ability and capacity to be unique for particular performance 
systems (Holton, 2000). Together, these drivers and outcomes describe the cause and 
effect relationships in organizations (Kaplan & Norton, 1996), which implies that drivers 
should predict future outcomes. 

Their theory fits well into the theory of HRD effectiveness, where HRD effectiveness is 
conceived as the extent to which HRD goals and objectives are achieved. (Wognum, 
1999). Following the school-effectiveness theory (Scheerens & Bosker, 1997) this theory 
points as well to a means-goals ordening between the ultimate criteria and the supportive, 
effectiveness enhancing criteria. The HRD effects can then be seen as ultimate criteria of 
HRD effectiveness, like Kaplan and Norton’s output measures. Criteria, such as the HRD 
process itself, resource acquisition, adaptability to the environment, or the ability, 
satisfaction and motivation of emplyees, are seen as effectiveness enhancing criteria 
(Scheerens & Bosker, 1997; Wognum, 1999), just like Kaplan and Norton’s performance 
drivers. 

Various authors state that the outputs of HRD can be measured at several levels. In 
general, these are the learning (effects on knowledge, skills and attitudes), behavior 
(effects on working behavior of individual employees), and results levels (effects on the 
performance results of individuals, groups, departments or the company). These multi- 
leveled outcomes fit within the multi-level theorie, which, according to Klein, Tosi & 
Cannella (1999) integrates the micro domain’s focus on individuals and groups with the 
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macrodomain’s focus on organizations, environment and strategy. This will results in a 
better understanding of the influence of individuals on the organization and, vice versa, 
the organizational influence on individuals’ actions and perceptions. Following this 
multi-level approach, it may become clear that the context of a specific organization will 
have an impact on the achieving of HRD performance or effectiveness. It is often 
assumed that there is a close link between characteristics of this context and the way in 
which organizational processes evolve. Deviating from an appropriate model, 
contingency theorists maintain, creates a lower degree of organizational effectiveness 
(Khandwalla, 1977; Van de Ven & Drazin, 1985). From this perspective, the only way to 
assess the effectiveness of HRD interventions is to include the interaction with key 
components of the organizational system (Hannigan, Donovan, Holton c.s., 2000). HRD 
interventions will probably be less effective if HRD structures and processes do not 
match their contextual conditions. 



Factors impeding or enhancing HRD effectiveness 
Strategic HRD Aligning 

Following the theories described in the former section, evaluating alternative areas is 
necessary, as a means of inteipreting the effects of HRD activities. This is also supported 
by Hannigan c.s. (2000), when they state that new research is needed that takes a more 
exploratory look at the decision and influence process that managers use to make 
decisions about training. This HRD decision-making process can be seen as congruent to 
strategic planning for HRD or HRD policymaking and strategy formation. In all these 
processes systematically considered decisions are made on actions and resources to 
accompish HRD goals, within certain environmental circumstances. In organizational 
effectiveness thinking, goals can be seen as the major defining characteristics of the 
effectiveness concept itself and have to be shared among the members of the 
organizations. (Scheerens & Bosker, 1997). 

Mintzberg states, that strategic planning cannot be synonymous with strategy 
formation and certainly not with effectiveness in that process (Mintzberg, 1994, p.29). 
Strategy formation in his view encompasses the meaning of strategy as a plan based on a 
certain intended and/or emergent pattern, while strategic planning refers to a more 
inflexible, rational and formalized procedure, following a sequential, linear and 
hierarchical order, mainly top-down. This is probably based on a more traditional view 
on strategic planning, holding in general the idea that strategic plans have been developed 
by a planning staff at top management level. This view has some shortcomings. It can 
destroy commitment, narrow a company’s vision, discourage change, and breed an 
atmosphere of politics. This especially becomes clear amid more dynamic external and 
more complex internal environments (Rothwell & Kazenas, 1994). Several authors, 
including Mintzberg (1994) and Galagan (1997), distance themselves from the traditional 
approach maintaining that what is needed for all those involved, within and outside the 
organisation, is to participate in this process in order to create commitment among 
employees and to achieve a successful implementation of policy. Strategic planning then 
implies that an organization have to pay attention to all relevant stakeholders when 
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developing organizational goals and means to reach these goals within certain 
environmental circumstances. 

A more recent view on strategic planning for HRD, which includes the 
participation of all relevant stakeholders, runs as follows: Strategic HRD planning - or 
strategic HRD aligning - is a dynamic and interactive process in which, as part of an 
ongoing and future company policy, HRD goals and objectives are formulated 
concerning employees’and company development using targeted learning processes of 
improvement and innovation (Wognum, 1999). The word ‘strategic’ emphasises the 
company perspective and connects the link between HRD and the organisational goals 
and objectives: According to the more traditional view, policymaking or planning 
processes are typically sequential, linear and mainly top-down. Less traditional strategic 
planning deviate from this, implying that goals and objectives are drawn up according to 
sufficient information and in consultation with all those involved. Based on rational 
policymaking models and on features of decision-making models, the process of strategic 
HRD aligning is characterized by four aspects: participation, information, formalization, 
and decision-making. Participation means the involvement of participants at the various 
organization levels in the aligning process. Information refers to the data needed to gain 
more insight in those company's situations which may call for HRD interventions. 
Formalization refers to the more or less formal consultative structures and information 
gathering procedures in the aligning process. Decision-making is concerned with the 
strategic choices of the aligning process, mainly the choice for HRD goals and objectives 
the HRD programmes are intended to. By paying attention to these four aspects HRD 
aligning will enhance HRD effectiveness. 

Contextual factors 

Literature indicates a number of contextual characteristics exerting influence on 
organizational effectiveness. Considering the HRD perspective, those characteristics refer 
to the HRD function, which is usually shaped into an HRD department, and the 
organizational context in which the function is embedded (Mintzberg, 1983; Tjepkema & 
Wognum, 1999). This environment can be divided into two parts: the company in which 
the HRD function is embedded, and the company’s external environment the so-called 
macro-environment. Macro-environment characteristics will be reflected in the kind of 
problems or developments with which the company is faced. Such problems serve as 
starting points for HRD interventions Therefore, macro-environment characteristics are 
not a direct object of study but act more indirectly by considering the problem that serves 
as a starting point for HRD interventions. This problem can be conceived as a 
characteristic of the company. 

The size and structure of the company, its HRD climate, the economic sector the 
company belongs to, and the degree of company's innovation are company characteristics 
that, according to literature (among others: Wognum, 1998; Mulder, Akkerman & 
Bentvelsen, 1989; Wexley and Latham, 1991; Useem, 1993), are supposed to have an 
impact on HRD effectiveness. The structure and position of the HRD department, the 
degree of innovation, and the form the HRD program takes, and also tranfer conditions 
are characteristics of the HRD function expected to influence HRD effectiveness (e.g. 
London, 1989; Mintzberg, 1983; Thijssen & Den Ouden, 1993). 
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The connection between all these elements and HRD effectiveness is visualized in the 
research model depicted in Figure 1 . 




Figure 1: Research Model 



Method 

This paper is based on data derived from a survey among Dutch companies. These 
companies were selected from a national database from the Association of Chambers of 
Commerce. 

Sample. Two company related contextual factors were used as selection criteria: the size 
of the company, and the economic sector to which the company belongs. Following these 
criteria, 44 companies with more than 500 employees were selected from the 
aforementioned database: 1 1 from the industrial sector and 33 from the financial and 
commercial services sector. A non-response analysis showed no significant differences 
between the non-response group and the 44 companies in the study, concerning such 
contextual variables as size and structure of the company and structure and position of the 
HRD function, hi each company one HRD program was selected (from the two 
frequently recurring fields of automation and social skills). This resulted in 23 selecting 
automation and 21 social skills programs. In order to define the effectiveness of the 
selected HRD programs, four categories of respondents within the company were 
selected: the HRD company representative, a maximum of 15 HRD participants, their 
supervising manager and (if present) their subordinates. All groups can be considered as 
having an interest in the results of the HRD programs (so-called stakeholders). 

Data collection. Data were collected in 1997, using four comparable questionnaires sent 
to 767 representatives of the four groups of stakeholders (44 HRD company 
representatives, 357 HRD participants, 242 of their supervising managers, and 124 of 
their subordinates). The questionnaires were designed to collect information on variables 
derived from the aforementioned theoretical framework. They comprised groups of 
questions and statements. Statements focused on respondents’ opinions on the strategic 
HRD aligning process (22 in total: five on participation, five on information, five on the 
formal nature, and seven on the decision-making process) and perceived effectiveness (9 
statements). In order to compare the effects of the 44 HRD programmes, the effectiveness 
as perceived by respondents was the object of study in stead of the real effectiveness. 
This is based on the assumption that the importance given to effectiveness later is every 
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bit as important as actual HRD effectiveness (Ford & Noe, 1987). Emphasis on 
perceptions is also allowed by the formerly mentioned multilevel theory, because the use 
of this theory will also results in a better understanding of the organizational influence 
on individuals’ actions and perceptions (Klein, Tosi & Cannella, 1999). Statements had 
answers on a 5-point scale, running from 1 (totally disagree/not al all) to 5 (totally agree, 
completely) and a possible 6 score (for unknown, no idea, or not relevant). 

Additional questions in the questionnaire for HRD company representatives 
concerned the company itself and HRD related factors. These included questions on the 
kind of problem that was a starting point for HRD (divided into three categories: to 
resolve a problem concerning employee, department or company performance; to 
improve certain working practices; or to change or renew the company situation), on the 
structure of the company and HRD function, on important innovations occurring since 
1990 within the company as well as in the HRD function, on the design of the selected 
program, and on conditions made for transferring of the learning results into working 
practice. The questions had both pre-coded and open answers. Twelve statements focused 
on company HRD climate and comprised, among others, statements on a respondent' s 
perception of management attitude towards training. These statements also had answers 
on a five-point scale. 

Response. For the automation response group 752 questionnaires were sent out (to 23 
HRD company representatives, 320 trainees, 277 managers, 132 subordinates). For the 
‘social skills’ group 740 questionnaires were sent out (to 21 HRD company 
representatives, 292 trainees, 25 1 managers, 176 subordinates). For both groups, all HRD 
company representatives filled in the questionnaire. The response rates for the other 
groups of respondents are for the ‘social skills’group: 63% of the trainees, 49% 
managers, and 49% subordinates, and for the group of companies with automation 
programmes: 54% of the trainees, 43% of the managers, and 29% of the subordinates. 

Analysis. To arrive at answers to the research questions descriptive statistics, cross- 
tabulations, chi-square analysis and one-way analysis of variance were used for the 
analysis of the data. A factor analysis was used to assess the construct validity of the 
scales. As a consequence of multilevel theory as described in a former section, and of the 
two-stage sampling design (first a sample of companies, then a sample of respondents 
within each company, which resulted in a hierarchical, nested datastructure) multilevel 
statistical models are used for the data analysis (cf. Bryk & Raudenbush, 1992; Goldstein, 
Rasbach, Plewis, et all, 1998) . These models allow us to make statistical inferences both at 
individual respondent (sample size 767) and company (sample size 44) levels. 



Results 

The study was intended to provide insight into which factors impede or enhance the 
achieving of improved performance. 

The impact of strategic HRD Aligning on HRD effectiveness. In order to study the 
impact of strategic HRD aligning on HRD effectiveness two scales were constructed, 
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'perceived strategic HRD aligning' and 'perceived HRD effectiveness'. A factor-analysis 
was used for the 22 items of strategic aligning to assess its construct validity. Instead of 
the expected four factors, six emerged from the analysis with varimax rotation covering 
all aspects of strategic aligning. A one-factor analysis was then performed to find out 
whether the construct of strategic aligning was one-dimensional instead of 
multidimensional. Five items with factor loading below .40 and with item-rest 
correlations below .30 were dropped for further analysis. Reliability of the remaining 17 
items of the opinion scale ‘perceived quality of strategic HRD aligning’ led to an alpha of 
.89. Analysis of the item content proved that all four aspects of strategic HRD aligning 
were represented in the constructed scale. This result justifies the conclusion that the four 
aspects are significant in connection. Also an opinion scale was constructed on the basis 
of nine items related to perceived HRD effectiveness at all three levels of HRD effects. 
The psychometric quality of this scale was studied with a one-factor analysis and with 
Cronbach’s alpha. The scale proved to be reliable. No items were excluded and ot=.89. 

Before analysing the impact of strategic HRD aligning on HRD effectiveness, the 
representativeness of the strategic aligning scale has to be considered, because this scale 
is based on 17 items and 543 valid cases; 224 cases were not included due to a high level 
of item non-response. Analysis showed that the results based on 543 cases are 
representative. Respondents whose opinion on the strategic aligning process is included 
in the analysis do not differ significantly in their opinion on HRD effectiveness (M=3.52, 
SD=.70) from those who are excluded in the strategic aligning scale (M=3.54, SD=.71) 
(t=.24, df=537, p>.05). 

Following the multilevel analyse strategy, as outlined above, model 0 was first 
constructed (see Table 1). This step was important to decide how much variance is 
explained by differences within and between companies and to find which variables 
account for this variance and to what extent. The results under model 0 indicate that the 
average perceived effectiveness score (the intercept) was 3.52, with a total variance of .49 
(.097 + .393), of which 20% ((.097/.49)*100%) is variation between companies and 80% 
is variation between respondents within companies. The results under model 1 (Table 1) 
indicate that supervising managers, subordinates and HRD company representatives 
estimate higher effects than HRD participants do. Supervising managers perceive HRD 
effectiveness .11 higher than HRD participants do; subordinates .06; and HRD 
representatives .25. These differences are significant for managers and HRD company 
representatives. One percent of the variance in perceived effectiveness is explained by the 
groups of respondents. This is not a significant model improvement (p>.05). 

The company score for stakeholders' opinion on strategic HRD aligning was 
constructed by first computing the mean of the deviation scores of each group of 
respondents and then calculating the mean of these four group scores. The results are 
presented in model 2 of Table 1. The data show that stakeholders' opinion on strategic 
HRD aligning within companies has a significant effect of .49 on perceived HRD 
effectiveness. The between companies effect of this so-called aligning perception is .47 
(also significant at the .10 level one-tailed). The strategic aligning perception accounts for 
22% of the variation in effectiveness scores (l-(.075+.304)/(.094+.389)*100%). The 
difference in deviation between model 1 and model 2a is 108.519 (884.932-776.413), 
with two degrees of freedom. This % 2 analysis indicates a significant difference (p=.00). 




7 



8 



This proved stakeholders' perception on strategic HRD aligning is an important predictor 
of perceived HRD effectiveness. The perceived effectiveness is even extra positive, as the 
mean company score on perceived HRD aligning is higher. 

The problem of cognitive dissonance. It is conceivable that respondents may have 
avoided cognitive dissonance (Etgen & Rosen, 1993; Festinger & Aronson, 1997), 
meaning that it is not very likely they will rate a course as effective while at the same 
time rating the strategic aligning process as poor. Multilevel statistical models are used to 
handle this problem. If avoiding cognitive dissonance affects the estimate of the 
association between stakeholders' opinion on strategic HRD aligning and effectiveness, 
this relates to the within-regression coefficient, but not the degree the between-company 
regression coefficient exceeds this within-company regression coefficient. There is no 
psychological explanation for a possible extra effect of the company mean for strategic 
aligning on perceived HRD effectiveness over and above the individual effects. For the 
individual level effect, the psychological explanation of avoiding cognitive dissonance 
may hold, but not for the so-called contextual effect. However, a study concerning the 
possible presence of cognitive dissonance has to start from company scores which are 
computed from the scores of individual respondents, regardless to which group they 
belong, instead of the mean scores of the four respondent groups, as was clarified before 

Table 1: Effects ( and standard errors) of strategic aligning on (perceived) HRD 
effectiveness (with respondent group of HRD participants as contrast group) (number of 
respondents = 440; number of companies = 44) 







model 0 




model 1 




model 2a 


Intercept 




3.52 


(.06) 


3.46 


(.06) 


1.83 


(.16) 


supervising managers 
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explained variance (R 2 ) 
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*) significant at a<.10 (two-tailed) 

**) significant at ac.l 0 (one-tailed) 



The results of this computing procedure show that stakeholders' opinion on strategic 
HRD aligning has an effect of .49 on perceived effectiveness within companies and that 
for the between effect the opinion on strategic HRD aligning is .63, which is considerably 
higher than .49. This extra effect may not only be explained by avoiding cognitive 
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dissonance, but must also be ascribed to the strategic HRD aligning process within 
companies (see Wognum & Lam, 2000). 

The impact of organization and HRD related factors on HRD Effectiveness. For 
exploring the impact of organization and HRD related factors, the variables size and 
structure of the company, degree of innovation of the company, structure and position of 
the HRD department, degree of innovation of the HRD department, design of HRD 
programs, conditions made for the transfer of learning results, and company HRD climate 
are separately brought into Model 2 (see Wognum, 2000). The variable economic sector 
only served as a selection criterium. It was not brought into the analysis since no 
significant effect on perceived HRD effectiveness was found concerning this 
characteristic (Wognum, 1999). For the factors size and structure of the organization, 
structure of the HRD function, and transfer conditions, no differences in perceived HRD 
effectiveness were observed. A link was found, however, between the following 
organization and HRD related factors and perceived HRD effectiveness. 

- Problem that served as starting Point for HRD. 

In interaction with the type of HRD program (automation or social skills) the problem 
that served as starting point for HRD proved to have a positive effect on perceived HRD 
effectiveness. If the actual performance of employees, departments or the entire company 
is starting point for HRD interventions, then the impact on perceived HRD effectiveness 
was significant lower for social skills programs then for automation programs. It was 
found that the problem that served as starting point for HRD has no significant effect on 
perceived HRD effectiveness in itself. 

- HRD Climate and Position of the HRD Department 

It proved that HRD climate correlates negatively with HRD effectiveness. No interaction 
effects were found between HRD climate and type of HRD program, indicating that the 
effect of HRD climate on perceived effectiveness is negative, regardless whether it 
concerns automation or social skills programs. 

The same applies to the position of the HRD department when this is not placed 
in the management line, but is part of a staff department, for instance, the company’s 
personnel department. The effect on perceived HRD effectiveness is significant lower 
where the HRD department is part of a staff department, than when it has 'other position 
than a staff department', which is significant. No interaction effect between 'position of 
the HRD department' and 'type of HRD program' were found, implying that the perceived 
HRD effectiveness score in companies with an HRD department as part of a staff 
department is significantly lower than in companies with an HRD department positioned 
as other than a staff department, no matter if it concerns automation or social skills 
programs. 



- The Form the HRD Program takes 

More than 56% of the 44 HRD programs studied were tailor-made, 25% were standard, 
while about 18% had another form, usually an adaptation of a standard program to a 
specific company situation. The form of the HRD program proved to correlate negatively 
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with perceived HRD effectiveness. When compared to standard and other kind of ‘off- 
the-shelf programs, tailor-made or customized programs result in a significant lower 
level of effectiveness. No significant effects were found when interacting the variable 
'form of HRD program' with the variable 'type of HRD program'. This means that the 
perceived effectiveness of tailor-made programs are less than other program forms, 
irrespective of whether they are geared to automation or social skills. 



Conclusion 

Based on the results it may be concluded that many factors proved to have an impact on 
HRD effectiveness. One of the findings showed that Strategic HRD Aligning is positively 
correlated with HRD effectiveness. This may lead to the conclusion that the strategic 
HRD aligning or decisionmaking process is an important intermediate factor in 
enhancing (or impeding) the impact of HRD on company performances. Also some 
environmental elements, like HRD climate, position of the HRD function and the form of 
the HRD program, turned out to be important effectiveness influencing factors. 

It is now possible to start the building of a model for HRD performance or HRD 
effectiveness, and visualize the connection between all elements. This can be done in 
several ways (see for instance Rossi & Freeman, 1993); Van Dellen, 1994; 
Bergenhenegouwen & Vooithuis, 1995; Holton, 1996). Based on the aforementioned 
theories and research findings, here the choice is made for an HRD effectiveness model 
derived from the integrated, multilevel model of school effectiveness, which is based on a 
review of instructional and school effectiveness literature (Scheerens & Bosker, 1997). 
Following this model and based on the information in the former sections, the main 
characteristics of the HRD effectiveness model (see Figure 2) are as follows: 

• According to a basic systems model of an organizationally and contextually 
embedded production process, the antecedent conditions are classified in terms of 
inputs, processes and contexts of HRD; 

• The model has a multilevel structure, where HRD function is embedded in 
organizations (strategic HRD aligning process), and individual employees who want 
to develop themselves are embedded in the HRD function (HRD process) 

In the study depicted here some empirical indications were found for certain relations in 
this model. The findings give some insights into ways to improve the effectiveness of 
HRD programs and other learning interventions. Much more research is needed, 
however, to discover more predictors of HRD effectiveness so that the effectiveness 
model can then be fleshed out and provide us with an adequate explanation for the 
effectiveness of company HRD programmes. Further research also needs to focus on the 
aligning processes in less traditional training situations. Due to the recent trend for less 
traditional and more informal HRD interventions than the ones mentioned in this survey, 
research is needed into the way the aligning process in more informal and on-the-job 
training and learning situations can be approached. 
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Figure 2: A model of HRD Effectiveness 
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